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Summary. 

Research  has  been  undertaken  on  visual  attention  and  short-term  visual 
memory  (VSTM) .  One  set  of  experiments  has  concerned  the  role  that  depth 
information  might  play  in  retrieval  from  VSTM,  using  an  iconic  memory 
paradigm.  Another  set  concerns  development  of  the  attention-gating  model, 
using  the  attentionrshift  RSVP  paradigm.  Finally,  we  are  studying  some 
effects  of  attention  and  visual  imagery  on  visual  acuity. 

(A)  Visual  attention  and  depth. 

Work  with  Dr.  Charles  Tijus  of  University  of  Paris  8,  who  is  visiting 
here  (Oct  88  to  Sept  89)  as  a  Post-doc,  has  concentrated  on  the  relevant  of 
depth  information  for  visual  memory.  Previous  work  in  visual  memory  has 
typically  employed  2-D  arrangements  of  letters  and  numerals.  Our  work  is 
directed  towards  finding  out  whether  providing  depth  cues  will  enhance 
performance.  The  work  is  essentially  exploratory. 

Depth  is  manipulated  in  two  ways;  by  providing  perspective,  and  by 
providing  stereo.  In  the  perspective  arrangement,  letters  designed  to 
appear  further  away  are  drawn  smaller  than,  and  above,  those  designed  to 
appear  close  up.  Fixation  is  requested  just  above  the  top  row  of  smaller 
characters,  so  that  the  smallest  letters  are  presented  closest  to  the  fovea 
where  acuity  is  highest.  In  this  way  accuracy  of  report  does  not  vary  too 
much  with  the  row  being  probed.  In  the  stereo  arrangement,  the  subject 
views  the  screen  through  a  prism  arrangement  in  front  of  the  left  eye. 
Cardboard  masks  direct  the  left  half  of  the  screen  to  the  left  eye,  and  the 
right  half  of  the  screen  to  the  right  eye.  The  subject  rotates  the  prism  so 
as  to  align  the  two  images.  The  depth  of  each  row  is  specified  by  varying 
the  relative  disparity  of  the  row.  All  the  letters  are  the  same  size  in  the 
'stereo  alone'  condition,  and  the  subject  fixates  the  center  of  the 
display.  In  the  perspective+stereo  arrangement,  both  perspective  and  stereo 
is  provided,  and  fixation  is  again  at  the  top.  In  these  experiments, 
letters  appear  to  be  in  different  depth  planes  in  different  rows,  but 
letters  do  not  overlap.  Arrangements  in  which  letters  are  viewed  'through' 
each  other  are  also  planned. 

In  the  final,  standard  condition,  the  display  is  flat  (without  stereo)  and 
the  letters  are  all  the  same  size.  Fixation  is  central. 

Visual  memory  is  tested  in  several  ways.  In  the  search  task,  the  subject 
must  search  the  stimulus  array  for  a  single  target.  In  the  partial-report 
task,  the  subject  will  be  asked  to  report  all  items  in  a  row.  The  identity 
of  the  target  or  of  the  row  is  either  known  before  the  trial,  or  revealed 
only  after  the  display  has  been  presented.  The  array  either  contains 
same-category  (e.g.,  letter-letter)  or  different  category  (eg,  letter- 
numeral)  distracting  characters.  Each  task  is  crossed  with  the  four  depth 
conditions. 

The  purpose  of  the  variation  of  category  is  to  establish  whether 
Sperling's  original  (I960)  result,  that  VIS  is  pre-categorical,  is  really 
correct.  In  Sperling's  work,  subjects  were  asked  to  write  down  items  that 
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were  either  all  in  the  sane  row  (standard  condition) ,  or  were  in  perhaps 
different  rows  but  belonged  to  the  same  category.  The  difficulty  that 
subjects  had  in  picking  by  category  was  used  to  infer  that  VIS  was 
essentially  visual  (ie,  precategorical) ;  or,  to  put  it  another  way,  that 
one  could  only  direct  one's  attention  to  a  part  of  the  remembered  array 
which  was  specified  by  location.  However,  the  difficulty  might  have  arisen 
from  the  task  of  having  to  report  items  that  were  in  fact  in  different 
rows,  and  thus  less  easy  to  organize  for  report.  In  the  search  task,  the 
subject  merely  has  to  report  whether  a  target  is  in  the  display,  and  its 
location;  this  task  is  exactly  the  same  whether  the  target  belongs  to  the 
same  category  as  the  distractors  or  not.  In  the  partial  report  task,  the 
subject  reports  the  items  without  being  asked  to  specify  their  location,  so 
that  report  organization  will  not  be  a  factor. 

Results  with  nine,  naive  subjects  in  the  search  task  show  that  both 
perspective  and  stereo  cues  help  performance.  A  long-term  study  with  three 
subjects  has  shown  that  the  affect  of  stereo  is  quite  constant  over  12 
sessions,  producing*  an  improvement  of  about  10%  in  accuracy  of  report  of 
location,  but  that  the  effect  of  perspective  increases  with  practice  to 
about  16%.  When  both  cues  are  provided,  it  seems  that  the  stronger  one 
wins;  that  is,  the  improvement  obtained  with  both  cues  is  roughly  equally 
to  the  greater  of  the  two  improvements  measured  separately. 

The  effect  of  stereo  is  easier  to  interpret  in  these  experiments  as  a 
pure  depth  effect,  because  the  spatial  structure  of  the  stimuli  (letter 
size  and  spacing)  is  the  same  with  and  without  stereo.  This  is  not  so  with 
the  letters  in  perspective,  however.  A  'flat'  control  for  perspective, 
using  the  same  set  of  letter  sizes  as  in  the  perspective  case,  but 
assigning  them  at  random  to  location,  was  therefore  run.  In  the  control, 
the  array  did  indeed  appear  flat,  but  the  large  letters  tended  to  laterally 
mask  the  smaller  letters  and  so  the  comparison  was  vitiated.  A  new  control, 
in  which  the  local  surroundings  of  characters  in  the  middle  of  the  display 
is  balanced  in  the  perspective  and  flat  conditions,  is  planned. 

Results  concerning  the  category  effect  are  so  far  incomplete.  With  the 
partial  report  method,  performance  dropped  about  10%  when  the  target  was  in 
the  same  category  as  the  distractors  (3  subjects) .  However,  preliminary 
indications  are  that  category  does  not  affect  search  for  a  single  target. 


(B)  Attention-shift  RSVP 

The  purpose  of  the  research  is  to  study  the  effects  of  shifting  1 

attention  on  the  perception  of  a  stream  of  characters  in  RSVP  (rapid  serial  ! 
visual  presentation) .  In  the  'attention-shift  paradigm'  being  used,  a  | 

subject  moves  his  attention  from  one  visual  display  (say,  left  of  fixation) 
to  a  second  RSVP  display  (say,  at  the  right) .  The  timing  of  the  attention 
shift  is  controlled  by  the  characteristics  of  the  first  display.  The 
effect  of  the  attention  shift  on  the  perception  of  the  second  RSVP  display 
is  studied  by  requiring  a  subject  to  report  characters  from  his  short-term 
memory  of  the  RSVP  display.  The  characters  used  in  the  RSVP  display  are 
letters,  numerals,  or  easily-codable  outline  geometric  forms,  which  are 
presented  in  a  stream,  one  after  another,  to  the  same  location  in  the 
visual  field.  The  main  finding  has  been  that  the  attention  shift  produces  a 
remarkable  temporal  order  illusion;  subjects  report  seeing  the  RSVP 
characters  in  an  order  different  from  their  presentation  order.  For 
example,  typically  the  fourth  character  after  the  attention-shift  cue  may 
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be  seen  as  occurring  first;  the  third,  as  next;  then  the  fifth,  then  the 
second;  and  so  on.  A  three  parameter  'attention-gating  model'  (AGM)  can 
capture  not  only  the  typical  order,  but  also  trial-to-trial  variability  in 
the  order  (Reeves  and  Sperling,  1986) .  The  current  work  involves  further 
testing  and  extending  the  AGM,  by  testing  the  effects  of  memory  load,  of 
item  category,  of  frame  size  (one  or  two  characters) ,  and  eye  movements. 

Memory  load 

In  developing  the  AGM,  we  assumed  that  attention  operated  to  gate  the 
entry  of  characters  from  the  RSVP  display  into  short-term  visual  memory; 
once  in  memory,  the  order  of  the  characters  was  unchanged.  The  'action'  in 
the  model  stems  from  the  bell-shaped  time  course  of  the  attention  shift, 
first  towards,  and  then  away  from,  the  RSVP  display.  Characters  occurring 
at  the  peak  of  the  bell  receive  maximum  attention  and  tend  to  be  reported 
as  seen  first;  both  earlier-  and  later-  characters  receive  less  attention 
and  tend  to  be  seen  as  later,  (falsely,  in  the  case  of  earlier  characters) . 
However,  in  an  important  theoretical  paper,  Grossberg  and  Stone  (1986) 
showed  that  a  model  in  which  the  attention  gate  was  virtually  box-car 
shaped,  but  assumed  internal  competitive  processes  within  VSTM,  could  also 
account  for  the  results.  They  termed  this  the  TOI  (temporal  order 
information)  model.  v 

One  way  in  which  the  models  may  be  compared  is  in  terms  of  their 
predictions  for  the  effects  of  memory  load.  Three  subjects  have  been  run  in 
an  experiment  in  which,  for  many  blocks  of  trials,  either  two,  or  four,  or 
six  characters  must  be  reported  on  each  trial.  The  first  question  was 
whether  the  data  of  the  first  two  reports  would  be  the  same  in  the  three 
report  tasks,  a  model-free  comparison.  The  second  question  is  whether 
parameters  derived  from  the  standard  recall-4  case  could  accurately  predict 
all  the  remaining  data.  The  answers  bear  on  the  comparison  of  AGM  with  TOI, 
because  as  the  models  make  different  predictions.  In  the  TOI,  the  strengths 
of  the  first  two  items  in  VSTM  should  depend  on  the  number  of  succeeding 
items,  because  each  incoming  item  tends  to  compete  with  resources  allocated 
to  the  earlier  items.  As  the  strengths  are  the  main  factors  determining 
perceived  order,  the  TOI  predicts  more  order  confusion  within  the  first  two 
recalls  as  the  memory  load  is  increased.  However,  the  AGM  predicts 
independence  of  memory  load  because  the  time-course  of  the  bell-like 
attention  shift  in  AGM  is  determined  by  the  first  display,  not  by  the 
second  RSVP  display.  Similarly,  the  parameters  derived  from  recall  4  should 
predict  recall  2  and  recall  6  data  according  to  the  AGM;  this  is  not  so  in 
TOI.  Thus,  independence  of  memory  load  in  both  experimental  tests  would 
favor  AGM  over  TOI. 

The  results  of  this  test  support  AGM.  The  model-free  comparison  showed 
no  systematic  variation  of  percent  order  error  with  recall  load,  as 
reported  in  the  first  6-month  period  and  confirmed  on  a  third  subject.  The 
model-dependent  fits  also  support  the  AGM  prediction  that  the  attention 
curve  should  not  depend  on  recall  load.  As  reported,  the  three  best-fitting 
AGM  parameters  (alpha,  tau,  and  sigma)  account  for  about  87%  of  the 
variance  when  fixed  for  each  subject  across  recall  load.  The  very  best  any 
one-dimensional  precedence  model  can  do  (the  Case-V  solution  in  Reeves  and 
Sperling,  1986,  with  every  precedence  strength  in  every  condition  allowed 
to  vary  freely)  averages  to  94%  of  the  variance,  so  the  very  constrained 
AGM  does  well  against  this  upper  limit.  (Moreover,  the  residual  variability 
appears  so  far  to  be  little  related  to  recall  load.)  Data  from  the  final 
subject  has  not  changed  this  picture. 
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Item  category 

A  second  line  of  attack  on  the  comparison  of  AGM  with  TOI,  concerns  the 
nature  of  the  items  entered  into  VSTM.  The  AGM  cannot  discriminate  between 
items,  at  it  is  'pre-categorical' ;  that  is,  the  gate  simply  operates  to 
permit  as-yet-unidentified  items  to  flow  into  VSTM.  However,  in  TOI  items 
compete  inside  a  memory  which  is  already  structured  by  resonant  feedback, 
so  that  items  within  the  same  class  (in  long-term  memory)  should  compete 
more  than  those  in  different  classes.  An  experiment  has  been  developed  to 
compare  the  models  by  interjecting  infrequent  'other-category',  or  'odd', 
stimuli  into  the  RSVP  stream.  So  far,  data  have  been  collected  using  a 
numeral  stream  with  the  odd  characters  being  Landolt  C's  of  different 
orientations.  It  has  already  been  shown  that  with  the  C's,  recalls  fit  the 
AGM  about  as  well  as.  with  numerals  (Reeves,  1987).  For  one  subject  (AR) , 
the  set  of  C's  is  well  learnt,  but  has  never  been  visually  presented  with 
the  set  of  numerals.  This  subject  has  learnt  a  coding  scheme  for  reporting 
C's  (pressing  the  key-pad  JLn  the  orientation  of  the  gap)  which  has  become 
easy  and  automatic  to  use.  The  preliminary  results  appear  to  support  the 
AGM,  because  the  'odd'  characters  seem  to  enter  VSTM  in  the  same  way  as  do 
the  numerals.  (Since  'odd'  characters  and  numerals  are  not  visual 
equivalent,  a  direct  comparison  of  order  accuracy  on  trials  with  and 
without  odd  characters  is  not  quite  appropriate.  Rather,  the  comparison  is 
made  between  reports  of  numerals  from  the  first  n-1  stimulus  positions  in 
trials  without  odd  characters  and  in  trials  with,  given  that  the  odd 
character  appeared  in  position  n.) 

This  is  a  weak  result,  in  that  TOI  is  not  committed  to  stating  the 
numerals  and  C's  must  be  in  different  classes.  On  the  other  hand,  finding  a 
difference  due  to  the  odd  character  would  be  a  direct  blow  to  AGM,  and 
support  TOI.  I  therefore  plan  to  continue  with  this  line  of  experiment,  and 
try  to  find  other  cases  in  which  the  evidence  may  favor  the  TOI.  My 
interest  is  not  merely  in  testing  the  two  models,  but  in  establishing 
whether  the  VSTM  assumed  by  both  models  can  indeed  be  treated  as  'pre- 
categorical'  (as  in  the  iconic  memory  experiments) . 

Frame  size  (single  or  paired  characters) 

Another  line  of  work  planned  in  the  grant  concerns  the  size  of  the  RSVP 
display.  This  work  will  become  possible  now  that  the  new  visual  display 
hardware  (for  rapid  point  plotting)  has  been  received. 


Attention  and  eye  movements 

A  different  line  of  work  concerns  the  comparison  of  attention  shifts 
with  and  without  saccadic  eye  movements.  The  main  finding  (with  the  eye- 
tracker)  is  that  there  is  little  difference  in  attention-shift  latency  or 
in  order  confusions  when  eye  movements  were  required  along  with  the 
attention  shift,  as  compared  to  eyes  still.  In  a  further  test,  the  subject 
is  required  to  shift  attention  to  the  RSVP  display  at  the  same  time  that  he 
executes  a  saccade  in  the  opposite  direction.  Two  subjects  have  now  been 
practiced  over  many  thousands  of  trials.  The  task  is  remarkably  difficult, 
but  is  now  accomplished  on  about  40%  (one  subject)  or  55%  (the  other)  of 
trials,  in  that  the  saccade  is  executed  in  the  appropriate  time  frame  (from 
.1  sec  after  the  cue  to  1  sec  after  the  cue);  the  saccade  is  accurate  to 
20%  of  the  distance  between  the  RSVP  streams;  and  that  a  numeral  report  is 
obtained  with  above  zero  confidence.  One  most  of  the  trials  in  which  an 
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opposite-direction  saccade  is  accomplished,  the  attention  shift  is  delayed, 
and  the  order  information  is  systematically  distorted  as  seen  in  the 
standard,  no  eye-movement  case.  However,  there  are  some  trials  in  which 
both  movements  appear  tc  occur  together;  in  these  cases  the  order 
information  seems  to  be  further  scrambled.  I  propose  to  run  the  subjects  in 
yet  further  sessions  in  order  to  collect  of  these  latter  types  of  trials 
(if  indeed  the  distinction  turns  out  to  be  really  warranted),  because  it  is 
of  considerable  interest  to  discover  whether  attention  shifts  and  saccades 
can  become  independent  of  each  other  after  sufficient  learning. 


(C)  Attention,  acuity,  and  the  'Perky  effect'. 

A  final  line  of  work  has  been  undertaken  by  the  PI  and  by  Catherine 
Lemley,  who  was  supported  by.  the  prior  grant  and  has  been  supported  part- 
time  on  this  grant-*  She  has  studied  the  effects  of  manipulating  two 
cognitive  variables,  attention  and  visual  imagery,  on  visual  acuity.  The 
first  paper,  on  visual  imagery  effects,  is  (still)  in  press  with 
Perception.  In  further  work,  we  have  discovered  a  complex  interaction 
between  imagery  and  attention .< In  one  manipulation,  in  which  attention  is 
distracted  from  the  acuity  task  by  a  winking  light  which  must  be  monitored, 
the  normal  effect  of  imagery  in  reducing  acuity  (the  'Perky'  effect)  is 
virtually  eliminated.  However,  when  attention  is  manipulated  by  requiring 
attention  to  either  one,  or  two,  possible  target  locations  in  each  trial 
block,  attention  and  imagery  have  independent  effects  on  acuity.  Since  we 
had  no  reason  to  expect  that  these  two  different  ways  of  manipulating 
attention  would  yield  different  results,  we  have  been  pursuing  an 
investigation  of  these  issues.  We  have  established  that  retinal  location 
(foveal  versus  parafoveal)  and  stimulus  duration,  both  of  which  had 
differed  across  the  two  original  attention  experiments,  do  not  account  for 
the  difference  in  the  results. 


EQUIPMENT 

A  custom-built  high-speed  point  plotter  has  been  obtained  from  Gary 
Finlay  at  the  University  of  Alberta.  This  plotter  is  specifically  designed 
for  the  computer  system  (PDP  11)  and  display  (Hewlet  Packard  1332)  that  I 
use.  It  increases  plotting  speed  by  a  factor  of  20,  and  so  provides  a  major 
upgrade,  allowing  experiments  with  many  more  characters  or  with  more 
complex  displays.  Software,  including  a  driver,  has  been  developed  for  the 
PDP/ 11-  plotter  combination  by  Terry  Caelli  of  the  Psychology  Dept. , 
University  of  Alberta.  Caelli  has  donated  this  software  free  of  charge. 

Disk.  The  Winchester  disk  and  controller  for  the  11/23  system  (used  for 
the  attention-shift  experiments)  I  had  originally  budgeted  at  $4,840  was 
obtained  for  much  less  ($  2475  from  CDC  and  DILOG) .  Installing  this 
combination  turned  out  to  be  a  problem,  since  the  power  requirements  were 
larger  on  +12  volts  than  supplied  by  the  DEC  system,  and  then  because  even 
when  installed,  the  disk/controller  intermittently  takes  14  seconds  (!)  to 
perform  an  access.  An  engineer  at  DILOG  is  now  trying  to  run  this  problem 
down.  Thus,  although  research  went  well  in  the  first  9  to  10  months  of  the 
grant,  it  has  recently  been  stalled  by  computer  malfunctions.  Research  on 
the  11/73  has  been  set  back  by  10  weeks  (so  far)  by  a  power  supply  failure 


P?|  which  blew  the  processor  card  and  some  of  the  image  processing  cards;  the 

'  former  card  has  been  repaired  but  the  latter  cards  are  still  being  worked 

Jn;  on  by  the  manufacturer  (ITI).  Repairs  of  both  systems  have  had  to  be 


charged  to  the  grant,  but  testing  the  cards  here  and  re-installation  has 
been  done  by  technical  staff  paid  for  by  the  University. 


PERSONNEL 

Graduate  Research  Assistant.  During  the  summer,  I  supported  Chisato  Aoki 
on  the  grant. 

From  Oct  1,  I  have  supported  Charles  Albert  Tijus  as  a  Post-Doc.  He  has 
a  doctoral  degree  (troisieme  cycle) ,  but  is  aiming  for  the  higher  Doctor 
D'etat  degree  from  the  University  of  Paris.  He  has  received  additional 
support  from  a  Lavoisier  grant  to  bring  his  total  income  up  to  Post-Doc 
levels.  I  have  been  paying  him  standard  doctoral  level  student  stipend. 

TRAVEL.  - 

I  spent  one  week  (Aug  10-15)  in  Edmonton  with  Terry  Caelli  and  Walter 
Bischof,  learning  their  software  system  for  DEC  11/23  -HP  plotter 
controller  combination.  I  have  also  obtained  and  learnt  some  other 
software,  for  our  11/73-Purkinje  tracker  equipment. 


